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ABSTRACT 

Numerous  observational  and  modeling  studies  have 
revealed  a  wide  variety  of  atmospheric  flows  around, 
through  and  above  terrain  obstacles.  Most  such  studies, 
however,  have  considered  fairly  simple  terrain  such  as  an 
isolated  summit  or  an  infinite  perpendicular  barrier 
exposed  to  uniform  or  relatively  simple  atmospheric 
conditions.  Real  terrain  and  real  atmospheric  conditions 
are  considerably  more  complex  than  those  used  in  the 
above  mentioned  studies  and  the  resulting  atmospheric 
flow  is  even  more  complicated  and  diverse. 

This  paper  highlights  the  results  from  a  series  of  high 
resolution  (1.0  km  grid  spacing)  numerical  simulations 
using  the  National  Taiwan  University  (NTU)  /  Purdue 
model  for  a  variety  of  flow  situations  such  as  hydraulic 
jump,  lee  waves,  juxtaposition  of  supercritical  and  sub- 
critical  flows,  etc.  To  complete  these  computations  in  a 
reasonable  amount  of  time,  the  NTU/Purdue  model 
simulations  were  run  on  a  1024-node  Linux  Networx 
Evolocity  II  cluster  at  the  Army  Research  Laboratory 
(ARL)  Major  Shared  Resource  Center  (MSRC).  The 
parallelized  NTU/Purdue  model’s  scalability 
characteristics  were  evaluated  for  a  fixed  grid  size,  with 
the  number  of  processors  ranging  from  4  to  128;  the 
model  scales  very  well  up  to  at  least  128  processors  on 
the  ARL  MSRC’s  Linux  cluster. 

1.  INTRODUCTION 

Much  of  the  White  Sands  Missile  Range  (WSMR) 
lies  to  the  lee  of  the  Organ  and  San  Andres  Mountains  in 
southern  New  Mexico.  The  Organ  Mountains  include  a 
rugged  quasi-circular  1500  m  high  massif  with  a  diameter 
of  about  10  km.  It  is  connected  to  a  narrow  and 
remarkably  steep  1  to  1.5  km  SE-NW  oriented  ridge.  The 


ridge,  in  turn,  connects  to  the  approximately  100  km  long 
south-to-north  oriented  barrier  of  the  San  Andres 
Mountains.  Several  passes  and  one  additional  massif 
complicate  the  San  Andres  barrier.  Because  of  the 
variations  in  elevation,  it  is  possible  during  a  given 
situation  at  WSMR  to  have  both  sub-  and  super-critical 
flows  in  juxtaposition,  greatly  complicating  the  resulting 
flow  and  presenting  significant  challenges  to  numerical 
forecasting  models. 

While  many  U.  S.  Army  missions  are  significantly 
impacted  by  the  highly  variable  weather  conditions  in  and 
around  complex  terrain  such  as  at  WSMR,  the  Army’s 
ability  to  accurately  forecast  and  diagnose  such  conditions 
remains  limited.  To  better  understand  and  to  evaluate  and 
improve  the  capabilities  of  numerical  models  to  forecast 
the  effects  of  terrain  on  weather  conditions,  the  ARL 
collected  surface  data  from  five  instrumented  towers  (10 
m  in  height)  positioned  in  the  lee  of  the  Organ  Mountains 
during  the  first  three  months  of  2004.  In  addition,  data 
from  the  WSMR  Surface  Automated  Meteorological 
System  (SAMS)  and  other  nearby  surface  stations  such  as 
the  Remote  Automated  Weather  Stations  (RAWS)  and  the 
wind  profiling  radar  at  White  Sands  were  collected.  The 
total  data  set  enables  meso-(3  (2  to  20  km)  and  -y  (200  m 
to  2  km)  scale  depiction  of  the  wind  flow  in  the  lee  of  the 
Organ  Mountains;  this  is  augmented  in  the  vertical 
direction  using  horizontal  and  vertical  wind  component 
data  from  the  White  Sands  wind  profiling  radar. 

The  following  sections  provide  a  brief  description  of 
the  numerical  forecasting  model  employed,  comparisons 
of  the  numerical  results  with  the  observations,  details  of 
the  model’s  scalability  characteristics,  and  a  discussion  of 
the  potential  for  future  applications. 
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2.  NUMERICAL  FORECASTING  MODEL 

The  NTU/Purdue  nonhydrostatic  numerical  model, 
hereafter  referred  to  as  the  NTU/P  model,  has  been  under 
development  over  the  last  8  years  to  predict  atmospheric 
motions  and  conditions  for  both  the  mesoscale  (200  m  to 
200  km)  and  large  scale  turbulence  (20  m  to  200  m) 
scales.  The  model  explicitly  solves  the  fully  compressible 
nonhydrostatic  system  of  equations  (Hsu  and  Sun,  2001) 
and  builds  on  the  proven  success  of  a  preceding 
hydrostatic  numerical  model  (Chern,  1994,  Sun  et  al. 
1991,  Sun  and  Chern,  1993,  Haines  et  al.,  1997).  The 
vertical  coordinate  of  the  model  is  defined  as: 


Po(z)-Po(zttv) 

P  0  ^  ^’surface  )  Po^Zfop') 


(1) 


where  p0 ,  the  reference  atmosphere  pressure,  is  strictly  a 
function  of  height.  Although  this  vertical  coordinate 
appears  to  be  the  usual  O  -pressure  coordinate  used  in 
many  hydrostatic  models,  the  pressure  in  (1)  is  not  a 
function  of  time,  so  the  position  of  each  grid  point  is  fixed 
in  time.  Hence,  the  model  employs,  strictly  speaking,  a 
O  -z  coordinate.  The  advection  terms  are  calculated  with 
the  Sun  (1993)  advection  scheme.  The  diffusion  process 
is  parameterized  through  a  level  2.5  turbulence  scheme. 
Density  is  a  prognostic  variable  which  gives  the 
advantage  is  that  there  is  no  diabatic  term  in  the  density 
prognostic  equation. 

The  pressure  field  is  diagnosed  through  the  equation 
of  state  for  a  perfect  gas, 


p=pRT , 


(2) 


where  p  is  density,  T  is  temperature,  and  R  is  the  gas 
constant.  Equivalent  potential  temperature  (0e)  is  used  as 
the  prognostic  variable  in  the  heat  equation,  with  6e 
defined  as: 


0=0  + 


~<lv 


(3) 


where  0  is  potential  temperature,  cp  is  the  specific  heat  at 
constant  pressure,  qv  is  the  specific  humidity  of  water 
vapor,  and  Lv  is  the  latent  heat  of  vaporization.  The  total 
specific  humidity,  qw  =  qv  +  qi  (where  qt  is  the  liquid 
water  content)  is  also  a  semi-conservative  quantity  in  the 
absence  of  precipitation. 


velocity  also  staggered  (with  respect  to  each  other)  in  the 
vertical  direction.  The  model  uses  a  two-tier  forward- 
backward  solution  procedure  that  is  neutral  in  time  with 
respect  to  both  sound  waves  and  internal  gravity  waves. 
This  forward-backward  scheme  for  sound  and  gravity 
waves  means  that  the  new  solution  values  corresponding 
to  the  current  time  immediately  replace  those  of  the 
preceding  time  during  the  calculation  so  that  only  one 
storage  array  is  required.  Therefore,  this  scheme  uses 
only  half  of  the  memory  space  required  by  centered  finite 
difference  schemes  typically  employed  in  numerical 
forecasting  models.  In  addition,  this  scheme  produces  no 
computational  modes  (i.e.,  numerical  noise),  thus 
avoiding  the  need  for  a  time  filter.  The  compressible  set 
of  equations  admits  fast  sound  waves  necessitating  a 
small  time  step  for  their  solution.  The  consequences  of 
the  small  time  step  are  mitigated  through  the  use  of  a 
time-splitting  technique  (Gadd,  1978)  in  which  the  time 
integration  is  split  into  three  stages;  the  corresponding 
time  steps  depend  on  the  physical  time  scales  of  the 
calculated  terms  which  involve  advection,  sound  and 
other  fast  waves,  and  diffusion.  The  details  of  the 
numerical  scheme  are  presented  in  Hsu  and  Sun  (2001). 
The  NTU/P  model  software  package  has  been  fully 
parallelized  using  Message  Passing  Interface  (MPI) 
library  routines. 

3.  OBSERVATION  DETAILS 

From  January  into  March  2004  the  ARL  set  up  five 
instrumented  towers  (10  m  in  height)  in  the  lee  of  the 
Organ  Mountains.  Each  tower  measured  and  recorded  the 
wind  direction  and  wind  speed  at  a  height  of  10  m  above 
ground  level,  as  well  as  pressure,  humidity,  and 
temperature  at  a  height  of  2  m  above  the  ground.  The 
pressure  instruments  used  were  inter-compared,  yielding 
an  accuracy  of  +/-  10  pascals  or  better.  Meanwhile,  ARL 
also  collected  data  from  several  of  the  nearby  WSMR 
SAMS  and  RAWS  sites  to  provide  a  fairly  detailed 
picture  of  the  surface  air  flow  in  and  around  the  Organ 
Mountains.  During  the  three-month  observation  time 
period,  there  were  several  down- slope  wind  storms  and  a 
variety  of  blocked-flow  episodes.  Besides  the  collected 
surface  data,  we  obtained  the  vertical  and  horizontal  wind 
profiles  recorded  by  the  two  collocated  wind  profiling 
radars  at  White  Sands  that  are  situated  about  15-20  km 
downwind  of  the  mountains.  These  radar  systems  provide 
data  coverage  from  about  500  m  above  their  ground 
elevation  (4000  feet)  to  about  17  km  above  sea  level. 


The  fully  explicit  solution  system  employed  by  the 
NTU/P  model  is  simple,  memory-efficient,  and  very 
accurate  for  high  frequency  waves.  Thus,  it  can  provide  a 
good  comparison  basis  for  the  implementation  of  semi- 
implicit  or  other  more  time-efficient  schemes  to  ensure 
their  accuracy.  As  with  many  forecasting  models,  the 
Arakawa-C  grid  is  employed,  with  density  and  vertical 


4.  SIMULATION  RESULTS 

4.1  Lee  Waves  /  Hydraulic  Jump 

On  January  25,  2004,  a  down- slope  wind  storm 
occurred  along  the  lee  of  the  Organ  and  San  Andres 
Mountains;  it  included  the  formation  of  a  train  of  lee 
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waves  which  extended  downwind  across  the  Tularosa 
Basin  and  a  hydraulic  jump  just  to  the  lee  of  the 
mountains  that  was  well-observed  by  the  ARL  and  SAMS 
pressure  sensors.  These  storms  occur  occasionally  at 
White  Sands,  but  this  day’s  storm  was  especially 
noteworthy  because  the  surface  and  other  observations 
were  able  to  document  many  of  its  facets,  thus  enabling 
detailed  comparisons  with  the  numerical  model  results. 

To  simulate  this  case,  the  NTU/P  model’s  three- 
dimensional  (3D)  grid  was  set  up  with  a  vertical  spacing 
of  300  m  and  a  horizontal  resolution  of  1  km.  Digital 
Terrain  Elevation  Data  (DTED)  level  2  terrain  data  was 
used  to  generate  (via  bi-linear  interpolation)  a  201  x  201 
grid  (with  1  km  spacing)  for  a  200  km  x  200  km  region 
centered  just  to  the  east  of  the  San  Augustine  Pass. 
Initially,  when  the  NTU/P  model’s  terrain  smoothing 
algorithm  was  employed  to  eliminate  two-delta  x 
variations  and  to  smooth  the  terrain  at  the  periphery  of  the 
model  domain,  we  found  that  the  maximum  terrain 
heights  were  somewhat  less  than  the  actual  terrain.  By 
experimenting,  we  found  that  a  two  step  process  could 
result  in  maximum  terrain  heights  quite  close  to  the  actual 
values.  In  the  first  step,  each  initial  terrain  height  above 
1400  m  was  increased  by  an  amount  equal  to  40%  of  the 
difference  between  the  initial  terrain  height  and  1400  m. 
Then,  in  the  second  step,  the  resulting  combination  of 
adjusted  and  unadjusted  terrain  heights  was  smoothed.  Of 
course,  it  was  not  possible  to  obtain  the  maximum  height 
of  very  narrow  features  such  as  the  Organ  Mountains 
ridge,  but  as  can  be  seen  in  Fig.  1  the  maximum  height  of 
the  Organ  Mountains  (2720  m)  and  of  the  southern  massif 
of  the  San  Andres  Mountains  (2189  m)  are  well  replicated 
in  the  terrain  field  used  in  this  study.  At  the  same  time, 
the  horizontal  breadths  of  the  mountains  and  the  height  of 
the  Tularosa  basin  are  preserved  and  agree  extremely  well 
with  the  real  terrain. 

Figure  2  reveals  the  details  of  the  El  Paso,  Texas 
sounding  at  12Z  (1200  UCT)  on  25  January  2004.  Note 
the  strong  inversion  at  about  600  hPa  or  about  4000  m 
above  sea  level  (asl).  Calculated  values  of  the  Scorer 
Parameter  (Scorer,  1949)  are  shown  in  this  figure  for  the 
areas  below  and  above  the  inversion.  It  is  known  that  a 
significant  decrease  of  the  Scorer  Parameter  in  the  vertical 
when  gravity  waves  are  likely  should  result  in  lee  waves 
downstream  from  the  terrain.  The  model  results  and 
observations  all  strongly  support  the  occurrence  of  lee 
waves. 

Using  the  sounding  shown  in  Fig.  2,  along  with  the 
assumptions  of  a  no-slip  surface  and  no  Coriolis  force,  the 
NTU/P  model  was  initialized  and  integrated  for  three 
hours.  The  simulation  results  are  presented  in  Figs.  3 
through  6.  Figure  3  compares  the  model-predicted  wind 
field  with  the  recorded  observations  at  0800  local  time 
(1500  UCT)  on  January  25,  2004.  In  this  figure,  the 


Fig.  1:  NTU/P  model  domain  terrain  field  for  the  central 
part  (i:80-120;  j:80-120)  of  the  201x201  grid;  terrain 
heights  are  in  meters  above  sea  level  and  contours  are 
every  100  m. 


NTU/P  model’s  surface  wind  field  is  represented  by  the 
wind  vectors,  which  indicate  a  strong  down- slope  flow  on 
the  lee  side  of  the  Organ  and  San  Andres  Mountains.  In  a 
band  oriented  mainly  south-to-north  just  east  of  the 
mountains,  the  model’s  surface  winds  abruptly  lessen  and 
then  either  reverse  (from  W  to  E)  or  remain  considerably 
diminished;  this  hydraulic  jump  corresponds  to  the  strong 
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adverse  pressure  gradient  shown  by  the  pressure 
perturbations.  The  transitions  to  reversed  flow  appear  to 
the  North  East  of  the  higher  terrain  areas  such  as  the 
Organ  Mountains  and  the  higher  terrain  to  the  north  of  the 
WSMR  post.  The  reversed  regions  are  about  5  km  wide; 
to  the  east  of  the  reversed  region,  the  model  surface  winds 
return  to  westerly  but  are  not  as  strong  as  in  the  strong 
down-slope  flow  area  just  to  the  lee  of  the  Organ 
Mountains.  Still  farther  to  the  east,  about  20-25  km  east 
of  the  mountain  chain,  there  is  another  diminished  or 
reversed  wind  region  which  is  not  as  continuous  or  strong 
as  that  immediately  to  the  lee  of  the  mountains. 


20 

Fig.  3:  Predicted  surface  wind  field  from  the  NTU/P 
model  simulation  of  the  January  25  down-slope  wind 
storm  case  at  WSMR  (after  3  hours  of  model  integration); 
yellow  arrows  indicate  the  observed  wind  field. 

The  observed  winds  are  shown  by  the  yellow  arrows 
in  Fig.  3.  The  model’s  wind  field  agrees  quite  closely 
with  the  actual  wind  field,  except  that  the  observed  winds 
at  the  San  Augustine  Pass  are  stronger  than  the  model’s 
prediction.  This  pass  is  approximately  1  to  2  km  wide. 
While  the  model’s  grid  spacing  is  1  km,  its  effective 
resolution  is  no  better  than  5  km.  Hence,  we  expect  that  a 
numerical  model  will  require  a  grid  spacing  of  about  300- 
400  m  or  less  to  adequately  resolve  the  wind  flow  at  the 
San  Augustine  Pass. 

Figure  4  compares  the  model’s  predicted  surface 
nonhydrostatic  pressure  perturbations  with  the  observed 
pressure  perturbations  from  the  ARL  and  SAMS  pressure 


sensors.  The  colored  contour  lines  show  the  NTU/P 
model’s  3  hour  forecast  of  the  surface  pressure 
perturbations;  the  values  range  (approximately)  from  0  to 
-200  pascals  (-2  hPa).  There  is  a  band  of  maximum 
negative  perturbation  pressure  running  south-to-north 
along  the  lee  side  of  the  Organ  Mountains.  A  little  farther 
east,  the  model  shows  a  band  of  little  or  no  perturbation 
pressure.  Even  farther  east,  there  is  another  band  of 
negative  perturbation  pressure.  These  model-predicted 
bands  continue  across  the  Tularosa  basin  in  connection 
with  the  lee  waves. 


Fig.  4:  Predicted  surface  pressure  perturbations  (color 
contours,  in  pascals)  from  the  NTU/P  model  simulation  of 
the  January  25  down-slope  wind  storm  case  at  WSMR; 
bold  numbers  (in  black)  indicate  the  observed  pressure 
perturbations  from  the  ARL  and  SAMS  observations. 

The  observations  shown  in  Fig.  4  include  the  ARL 
instrumented  towers  and  the  SAMS  sites  (indicated  by  *) 
in  this  area.  The  observed  pressure  perturbations  were 
very  carefully  extracted  through  pressure  reduction  to  a 
common  datum  plane  at  1295  m.  With  the  exception  of 
the  observation  at  the  San  Augustine  Pass,  the  height  of 
this  plane  minimized  the  vertical  distance  over  which 
pressure  reduction  was  done  for  the  observations, 
consequently  minimizing  the  error  of  the  extracted 
pressure  perturbations. 

Except  for  the  San  Augustine  site,  the  model’s 
predicted  wind  field  and  pressure  perturbation  bands  are 
in  good  agreement  with  the  observations.  In  addition,  the 
model’s  predicted  pressure  perturbation  compares  well 
with  the  observation  from  the  Oro  Grande  Gate  SAMS 
which  is  located  far  to  the  east  of  the  area  shown  in  Fig.  4. 

Vertical  cross-sections  of  vertical  and  horizontal 
wind  components  and  the  pressure  perturbations  support 
the  occurrence  of  a  hydraulic  jump  in  conjunction  with 
the  wind  reversal  and  adverse  pressure  gradient  shown  in 
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Figs.  3  and  4.  Figure  5  shows  a  west  to  east  cross  section 
of  the  NTU/P  model’s  predicted  vertical  velocity  field  for 
0800  local  time  (1500  UCT)  on  January  25,  2004,  located 
north  of  the  WSMR  post,  approximately  intersecting  the 
San  Augustine  Pass  and,  more  importantly,  coinciding 
with  the  location  of  the  White  Sands  wind  profiling  radar 
(shown  by  the  dark  vertical  line  in  Fig.  5).  The  model 
prediction  shows  partially  trapped  lee  waves  extending 
eastward  across  the  Tularosa  Basin.  The  maximum 
vertical  velocities  are  greater  than  3  m/s.  Using  the 
Scorer  Parameters  calculated  from  the  El  Paso  sounding, 
the  theoretical  wavelength  of  13-14  km  compares  fairly 
closely  to  the  model-predicted  wavelength  of  around  14- 
15  km.  The  observed  lee  waves  are  well  represented  by 
the  model’s  surface  winds,  are  in  fairly  good  agreement 
with  the  wind  profiler  observations,  and  are  consistent 
with  a  decrease  (with  height)  in  the  Scorer  Parameter, 
defined  as: 


2  _Af2  1  d^U 

-  U2  U  dz 2  ’ 


(4) 


where  U  is  the  horizontal  velocity  (m/s)  and  N  is  the 
Brunt- Vaisala  frequency  (1/s). 


NTU/Purdue  Vertical  velocities 


Fig.  5:  NTU/P  model-predicted  vertical  velocity  field 
(m/s)  at  0800  local  time  on  January  25,  2004  along  a  west 
to  east  cross-section  that  intersects  with  the  location  of  the 
White  Sands  (vertical  black  line)  wind  profiling  radar. 

Figure  6  compares  the  NTU/P  model’s  predicted 
vertical  velocities  with  the  White  Sands  wind  profiler  data 
on  January  25,  during  approximately  the  same  time 
interval  (0700  -  0900  local  time).  In  this  figure,  the  y- 
axis  represents  the  height  (in  meters)  asl,  while  the  x-axis 


is  the  vertical  velocity  (in  m/s).  The  model’s  vertical 
velocity  profiles  are  shown  by  continuous  color  lines;  the 
corresponding  times  are  shown  in  the  legend  on  the  upper 
right  hand  side  of  this  figure.  The  model  shows  two 
peaks  in  vertical  velocity:  one  maximum  at  about  4500  m, 
and  the  second  at  about  12000  m.  The  model  predicted  a 
steady  drop  in  the  lower  peak’s  vertical  velocities  from 
0730  to  0830,  with  essentially  no  change  between  0830 
and  0900,  while  the  upper  peak’s  vertical  velocities  were 
steadier  over  this  same  time  period. 


Fig.  6:  Comparison  of  the  NTU/P  model-predicted 
vertical  velocity  profiles  with  the  observed  velocity 
profiles  at  the  White  Sands  wind  profiling  radar  location 
on  January  25,  2004. 

In  Fig.  6,  the  vertical  velocity  profiles  recorded  by 
the  White  Sands  wind  profiler  are  shown  by  the  black 
pluses  and  blue  pound  signs.  These  are  the  mean  values, 
respectively,  for  the  “0700-0800”  and  “0800-0900”  local 
time  periods;  what  is  shown  has  been  smoothed  by  using 
a  running  5  point  mean  value,  because  the  individual 
point  values  are  quite  noisy.  The  wind  profiler  is  believed 
to  have  an  accuracy  of  +/-  0.5  m/sec,  so  the  corresponding 
error  bars  were  included  for  each  observed  value.  The 
observations  show  that  the  lower  maximum’s  vertical 
velocities  decrease  going  from  hour  1  to  2  but  the  upper 
maximum’s  vertical  velocities  are  roughly  the  same.  It 
should  be  mentioned  that  the  profiler’s  winds  are  obtained 
from  processing  of  radial  components  along  three  beams 
that  are  not  quite  vertically  oriented  so  the  obtained  value 
is  probably  more  representative  of  the  vertical  velocity 
over  a  volume  of  space  rather  than  being  a  point 
measurement.  Despite  these  limitations,  the  NTU/P 
model’s  predictions  of  vertical  velocity  are  in  general 
agreement  with  the  observations  at  the  White  Sands 
profiler  site. 
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4.2  Blocking  Effects  /  Down-Slope  Winds 

The  blocking  effects  of  mountains  on  air  flow  have 
been  studied  for  years.  However,  most  studies  have  been 
conducted  with  either  an  idealized  mountain  or  for 
synoptic-meso scale  systems  due  to  the  difficulties  of 
collecting  very  high  resolution  data  or  even  developing  a 
reliable  forecasting  model.  Also,  until  recently, 
computational  limitations  have  prevented  the  numerical 
solution  of  this  high-resolution,  three-dimensional 
problem  in  a  reasonable  amount  of  time.  The  results  from 
this  study  indicate  that  the  high  resolution  NTU/P  model 
is  capable  of  reproducing  the  details  of  the  flow  in  very 
pronounced  terrain  under  different  prevailing  wind 
conditions.  Using  the  same  grid  and  terrain  data  as  in  the 
preceding  case,  the  model  was  initialized  with  the  El 
Paso,  Texas  sounding  at  12Z  on  January  19,  2004  (see 
Fig.  7)  and  integrated  for  three  hours. 


72364  EPZ  Santa  Teresa 


Fig.  7:  El  Paso,  TX  sounding  for  January  19,  2004  at  12Z. 

Because  of  the  variation  in  terrain  heights,  some  of 
the  areas  to  the  lee  of  the  Organ  Mountains  on  January  19 
are  subject  to  blocking,  while  the  flow  for  other  areas  is 
supercritical  and  down-slope  winds  develop.  The  model- 
predicted  overall  flow  situation  is  quite  complex,  as 
indicated  by  Fig.  8.  The  model-predicted  wind  field  is 
represented  by  the  colored  vectors.  The  low  level  wind 
field  is  quite  complex;  there  is  blocked  flow  (BF)  in  the 
lee  of  the  Organ  Mountains,  but  also  down- slope  wind 
(DW)  flow  on  the  south  side  of  the  Organs  and  to  the 
north  of  the  WSMR  post. 

The  Froude  Number  (defined  in  the  upper  right  hand 
corner  of  Fig.  8)  provides  a  good  measure  of  whether  air 
will  go  over  or  around  a  terrain  obstacle.  Because  of  the 
different  height  scales,  in  this  case  the  Froude  Number  is 
less  than  1  for  flow  trying  to  go  directly  over  the  Organ 


Mountains,  but  is  greater  than  1  elsewhere  so  the  flow  can 
go  over  the  terrain.  Hence,  we  have  a  juxtaposition  of  sub 
and  super-critical  flow,  which  helps  produce  the 
complexities  shown.  Note  that  the  observations  generally 
confirm  the  model  predictions,  although  the  observed 
winds  at  the  San  Augustine  Pass  are  stronger  than  those 
given  by  the  model. 


Froude  Number,  FN 
FN  =  U/Nh  where 
U  =  wind  speed 
N  =  Brunt-Vaisala 
Frequency 
h  =  terrain  height 


Figure  8:  Predicted  surface  wind  field  (color  vectors) 
from  the  NTU/P  model  simulation  for  January  19,  2004  at 
WSMR  (after  3  hours  of  model  integration);  black  arrows 
indicate  actual  observations  (10  m  above  the  ground). 

The  blocking  effect  becomes  more  pronounced  when 
the  Froude  Number  is  less  than  0.5,  so  we  decided  to 
investigate  the  case  for  a  5  m/s  westerly  prevailing  wind. 
Using  the  same  grid  and  terrain  data  as  in  the  preceding 
cases,  the  model  was  initialized  with  a  5  m/s  westerly 
wind,  uniform  stratification,  and  integrated  for  several 
hours.  Figure  9  shows  the  resulting  model-predicted 
surface  wind  field  and  surface  streamlines;  the  color 
contour  regions  in  this  figure  indicate  the  terrain  levels. 

In  Fig.  9(a)  we  can  see  that  the  surface  wind  is 
blocked  by  the  Organ  Mountains  and  produces  a  counter¬ 
gradient  flow  on  the  windward  side  (near  x  =  95,  y  =  95), 
stronger  wind  at  San  Augustine  Pass  (near  x  =  95  and  y  = 
102),  and  lee  vortices  (at  x  =  105,  and  y  =  35,  85),  as  well 
as  a  strong  down-slope  wind  on  the  lee  side,  all  of  which 
are  also  clearly  indicated  by  the  surface  streamlines  in 
Fig.  9(b). 

The  blocking  effects  of  the  WSMR  terrain, 
particularly  in  the  lee  of  the  Organ  Mountains,  have  been 
shown  in  SAMS  and  other  data  (Grove  and  Haines, 
2002).  Grove  and  Haines  noted  that  the  wind  flow  shown 
by  the  SAMS  stations  was  consistent  with  the  formation 
of  a  lee  vortex.  However,  the  number  of  SAMS  stations 
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(a)  surface  wind  field 


westerly  and  southwesterly  flow  cases  in  which  the 
Froude  Number  was  less  than  0.5.  During  the  January  to 
March  2004  observations,  fully  and  partially  blocked 
flow,  as  well  as  lee  waves  and  hydraulic  jumps  in  the  lee 
of  mountains,  were  recorded. 

5.  SCALABILITY  RESULTS 

As  indicated  in  Section  2,  the  NTU/P  model  software 
package  has  been  fully  parallelized  using  Message 
Passing  Interface  (MPI)  library  routines.  To  evaluate  the 
scalability  characteristics  of  this  advanced  forecasting 
model,  we  ran  the  case  described  in  Subsection  4.1 
(down-slope  wind  storm  case  at  WSMR  on  January  25, 
2004)  on  the  unclassified  1024-node  Linux  Networx 
Evolocity  II  cluster  at  the  ARL  MSRC.  Maintaining  a 
fixed  problem  size  of  four  million  grid  points 
(200x200x100),  we  varied  the  number  of  processors  from 
4  to  128  and  ran  the  test  case  for  three  hours  of  simulation 
time.  Table  1  shows  the  total  wall  clock  times,  speed-up 
factors  (relative  to  the  4-processor  case),  and  percentages 
of  optimum  speed-up  achieved  for  the  various  numbers  of 
processors.  As  the  table  indicates,  the  NTU/P  model 
scales  very  well  on  the  ARL  MSRC’s  Linux  cluster. 


(b)  streamlines 


Fig.  9:  NTU/P  model  predictions  of  (a)  surface  wind  field 
and  (b)  streamlines  for  the  case  of  an  initial  5  m/s 
westerly  prevailing  wind  at  WSMR  (after  2.5  hours  of 
model  integration);  terrain  levels  (m)  are  indicated  by 
color  contour  regions. 

near  the  Organ  Mountains  is  limited,  and  additional 
measurements  would  be  required  to  fully  reveal  the  actual 
flow.  The  kinds  of  blocking  predicted  by  the  NTU/P 
model  has  also  been  seen  in  the  observational  results  for 


Table  1.  Scalability  Results  for  NTU/P  Model 


Number  of 
Processors 

Wall 

Clock 

Time 

(S) 

Actual 

Speed¬ 

up 

Factor 

Optimum 

Speed-up 

Factor 

%  of 
Optimum 
Speed-up 

4 

78523 

reference 

- 

- 

8 

40287 

1.95 

2 

97.5 

16 

20925 

3.75 

4 

93.8 

32 

11114 

7.07 

8 

88.3 

64 

5998 

13.09 

16 

81.8 

128 

3658 

21.47 

32 

67.1 

CONCLUSIONS 

The  NTU/P  model  was  applied  to  several  well- 
observed  real  terrain  cases  for  the  Organ  and  San  Andres 
Mountains  in  southern  New  Mexico.  On  January  25, 
2004,  a  down-slope  wind  storm  in  the  lee  of  these 
mountains  was  accompanied  by  a  hydraulic  jump  and  lee 
waves,  all  of  which  were  well-observed.  The  NTU/P 
model  was  initialized  with  the  12Z  El  Paso  sounding  for 
January  25  and  run  for  several  hours.  As  shown,  the 
model  successfully  predicted  much  of  the  atmospheric 
phenomena  observed  on  that  day.  The  model  was  also 
applied  to  partially  and  fully  blocked  flow  conditions  on 
the  WSMR  terrain.  With  a  Froude  Number  of  less  than 
0.5,  the  model  shows  that  the  terrain  causes  pronounced 
blocking,  along  with  the  formation  of  a  lee  vortex.  The 
model-predicted  surface  wind  fields  are  consistent  with 
the  local  WSMR  surface  flows  observed  for  these  kinds 
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of  conditions.  On  January  19,  2004,  the  WSMR  surface 
observations  showed  super-  and  sub-critical  flows.  The 
NTU/P  model  was  initialized  with  the  12Z  El  Paso 
sounding  for  January  19  and  run  for  several  hours.  The 
model’s  predicted  wind  flow  agrees  fairly  well  with  the 
intricacies  of  the  complex  flow  that  were  actually 
observed. 

Knowing  the  surface  and  boundary  layer  wind  fields 
benefits  the  Army  in  several  important  ways.  The 
characterization  of  dispersion  and  diffusion  of  various 
biological/chemical  agents  in  the  battlefield  requires 
detailed  information  including  wind  speed,  wind 
direction,  and  the  variability  of  both,  all  of  which  can  be 
reliably  provided  by  high  resolution  numerical  modeling. 
Also,  some  munitions  are  significantly  affected  by  the 
low-level  wind  field  as  they  maneuver  in  acquiring  and 
engaging  their  target;  obtaining  a  reliable  forecast  of  the 
target  area  wind  field  will  allow  them  to  be  more 
optimally  aimed.  In  addition,  the  low-level  wind  and 
turbulence  fields  are  linked;  hence,  knowing  where  to 
avoid  severe  turbulence  and  adverse  winds  will  greatly 
benefit  Army  aviation  missions,  especially  those 
involving  helicopters,  unmanned  aerial  vehicles,  and  para- 
drop  operations.  Finally,  the  wind  environment  and 
embedded  turbulence  significantly  affect  acoustic 
propagation,  so  accurate  numerical  forecasts  of  wind  and 
turbulence  fields  will  help  acoustic  detection  and 
avoidance  programs. 

While  the  results  presented  in  this  paper  demonstrate 
the  feasibility  of  producing  high  resolution  forecasts  of 
complex  wind  flow  caused  by  and  occurring  in  proximity 
to  mountainous  terrain,  much  additional  work  remains  to 
be  done.  The  NTU/P  model  used  here  was  found  to  have 
high  scalability  characteristics,  so  that  with  the  model 
domain  used  in  this  paper,  it  is  possible  to  obtain  the 
forecast  results  over  21  times  faster  using  128  processors 
versus  use  of  only  4.  This  means  that  a  three  hour 
forecast  can  be  completed  in  about  an  hour.  If  a  semi- 
implicit  approach  is  implemented  to  solve  the  vertical 
momentum  equation,  and  higher  computational  efficiency 
is  attained,  it  should  be  possible  to  more  than  halve  the 
NTU/P  model’s  run  time.  The  ARL  MSRC  will  soon  be 
installing  a  new  Linux  Networx  Advanced  Technology 
Cluster  with  1122  dual-core  compute  nodes.  This 
significant  increase  in  computing  power  will  enable  us  to 
generate  more  detailed  weather  forecasts  using  even 
higher  resolution  grids  (e.g.,  100-m  grid  point  spacing, 


with  billions  of  grid  points)  for  regions  with  complex 
terrain.  This  will  help  us  determine  the  levels  of  detail 
and  accuracy  that  can  be  achieved  using  the  currently 
available  (state-of-the-art)  numerical  weather  forecasting 
models.  While  increased  grid  resolution  (i.e.,  smaller  grid 
spacing)  may  lead  to  more  accurate  forecasts,  it  may  also 
reveal  model  deficiencies  that  can  be  corrected,  or  at  least 
improved.  As  computing  power  continues  to  increase,  we 
anticipate  that  a  practical  numerical  weather  forecasting 
tool  will  eventually  be  developed  for  use  in  the  field.  The 
ultimate  goal  of  this  work  is  to  provide  Army 
commanders  with  accurate  assessments  of  meteorological 
conditions  in  the  battlefield  (in  near  real-time)  to  assist  in 
their  strategic  planning  and  decisions. 
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OBJECTIVE 


Evaluate  the  capabilities  of  a  nonhydrostatic  model  to  predict 
weather  conditions  in  complex  terrain  for  the  U.  S.  Army 
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Organ  Mountains,  NM 


NTU/Purdue  Nonhydrostatic  Model 


•  3-D  Fully  Explicit,  Non-Hydrostatic,  Compressible  Equations 
(Hsu  and  Sun,  Tellus  A  53  (3),  2001).  0^,  p,  qw,  u,  v,  w. 

•  Double  Forward-Backward  Time  Integration  procedure  for 
treating  both  sound  waves  and  internal  gravity  waves  .  Uses  the 
time-splitting  approach  for  the  gravity  wave  and  advective  steps. 

•  Uses  a  terrain  following  vertical  grid  that  is  consistent  with  the 
Purdue  Regional  Model,  so  it  can  use  already  developed  physical 
parameterizations  for  the  ground  surface,  microphysics,  and 
radiative  transfer. 

•  Solves  the  density  tendency  equation  instead  of  the  pressure 
tendency  equation;  no  diabatic  term. 


NTU/Purdue  Model  Domain  (WSMR) 


GrJrDS;  C0LA/IGE5 


2005-11-02-15:06 


Local  T errain  Details 


GrADS;  COLA/IGES 


2005-11-02-16:0 


Summary  of  NTU/Purdue  Model  Results 


Numerical  forecasts  of  wind  flow  were  generated  for  two 
different  dates:  01-25-2004  and  01-19-2004. 

NTU/Purdue  model  was  initialized  with  the  El  Paso  1 2Z 
soundings  from  those  dates,  and  then  integrated  for  3  -  4  hours  to 
obtain  numerical  forecasts  of  the  wind  flow. 

Model  results  were  compared  to  Meso-y  and  WSMR  Surface 
Atmosphere  Measuring  System  (SAMS)  wind  and  pressure 
observations,  the  White  Sands  Profiler,  and  other  data. 

For  the  01-25-2004  case,  both  the  model  and  observations 
indicate  strong  downslope  winds,  a  stationary  wave  train 
extending  downwind  from  the  mountains,  and  (apparently)  a 
hydraulic  jump. 

For  the  01-19-2004  case,  both  the  model  and  observations  show 
blocked  flow  in  the  lee  of  the  Organ  mountains  and  downslope 
flow  to  the  north  of  the  WSMR  post  area. 


El  Paso  Sounding  for  12  Z  on  January  25,  2004 
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Comparison  of  Model  Predictions  with  Observed  WSMR 
Pressure  Perturbations  at  0800  on  January  25,  2004 
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Comparison  of  Model  Predictions  with  Observed 
WSMR  10  m  Wind  Field  at  0800  on  January  25,  2004 
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Comparison  of  Model  Predictions  with  Observed 
WSMR  10  m  Wind  Field  at  0800  on  January  25,  2004 
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Model-Predicted  Vertical  Velocity  Field  at  y=102 


This  cross- 
section 
includes 
the  WSMR 
Wind  Profiler 
(dark  line).  The 
vertical 
velocities  (m / 
sec)  show  the 
trapped  lee 
waves 
extending 
eastward  away 
from  the 
mountains. 

The  altitude 
(m)  is  shown 
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edge  of  this 
figure 


NTU/Purdue  Model-Predicted  Vertical  Velocities 
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NTU/Purdue  model 
vertical  velocities:  ■  ■  ■ 1 
(connected  lines) 

Wind  Profiling 
Radar  observed 
vertical  velocities 
(+  and  # )  with  0.5 
m/s  error  bars. 

The  vertical  scale 
is  altitude  in 
meters  asl  . . 


Average  terrain 
height  upwind  . 

Elevation  of  Basin 
Floor 


Comparison  of  Model-Predicted  Vertical  Velocity  Profiles 
with  Observations  from  WSMR  Wind  Profiler  on  01-25-2004 
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GrADS:  CQLA/IGE5 
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El  Paso  Sounding  for  12  Z  on  January  19,  2004 
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Comparison  of  Model  Predictions  with  Observed 
WSMR  10  m  Wind  Field  at  0800  on  January  19,  2004 


Wind  Vectors: 
NTU /  Purdue- 
NH  Model 

Dark  Arrows:. 

Observed 

Winds 


The  low-level 
flow  shows  a 
lee  vortex  (L) 
east  of  the 
Mountains, 
blocked  flow 
(BF)  to  its  north, 
and  downslope 
wind  (DW)  on 
the  lee  north  of 
WSMR  post. 

All  are 

consistent  with 
the  local  flow’s 
Froude  number. 


Froude  Number,  FN 
FN  =  U/Nh  where 
U  =  wind  speed 
N  =  Brunt-Vaisala 
Frequency 
h  =  terrain  height 
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(O  /ZCL  Scalability  Results 
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NTU/P  Model  Scalability  for  Fixed  Problem  Size 


Number  of 
Processors 

Wall 
Clock 
Time  (s) 

Actual 

Speed-up 

Factor 

Optimum 

Speed-up 

Factor 

%of 

Optimum 

Speed-up 

4 

78523 

reference 

-- 

-- 

8 

40287 

1.95 

2 

97.5 

16 

20925 

3.75 

4 

93.8 

32 

11114 

7.07 

8 

88.3 

64 

5998 

13.09 

16 

81.8 

128 

3658 

21.47 

32 

67.1 

CONCLUSIONS 


The  NTU/Purdue  model  was  run  for  two  well-observed 
WSMR  cases. 

•  The  model  successfully  generated  a  hydraulic  jump  signature 
for  the  lee  wave  case. 

•  For  the  case  with  juxtaposition  of  super-  and  sub-critical  flow, 
the  model’s  wind  flow  agreed  fairly  well  with  the  observations. 


The  model  results  demonstrate  that  it  is  possible  to  forecast 
the  details  of  flow  in  complex  terrain. 


The  numerical  implementation  of  the  model  scales  fairly  well 
(up  to  128  processors)  for  the  cases  presented. 


/'j’l  Potential  Benefits/Applications 


Improved  forecasts  of  dispersion  and  diffusion  characteristics  of 
various  biological/chemical  agents  in  the  battlefield 

Improved  success  rate  for  Army  aviation  missions,  especially 
those  involving  helicopters,  unmanned  aerial  vehicles,  and  para- 
drop  operations 

Improved  aiming  of  munitions  that  are  significantly  affected  by 
the  low-level  wind  field 

Improved  characterization  of  acoustic  propagation  in  the 
battlefield 

Ultimate  Goal:  to  provide  Army  commanders  with  more  accurate 
assessments  of  meteorological  conditions  in  the  battlefield  (in 
near  real-time)  to  assist  in  their  strategic  planning  and  decisions 


